Enhancing Parameter-Free Frank Wolfe with an Extra Subproblem

Aiming at convex optimization under structural constraints, this work introduces and analyzes a variant of the Frank Wolfe (FW) algorithm termed ExtraFW. The distinct feature ExtraFW is pair gradients leveraged per iteration, thanks to which decision variable updated in prediction-correction (PC) format. Relying on no problem dependent parameters step sizes, convergence rate for general problems shown be ${\cal O}(\frac{1}{k})$, optimal sense matching lower bound number solved FW subproblems. However, merit its faster O}\big(\frac{1}{k^2} \big)$ class machine learning problems. Compared with other parameter-free variants that have rates same problems, has improved fine-grained analysis PC update. Numerical tests binary classification different sparsity-promoting constraints demonstrate empirical performance significantly better than FW, even Nesterov's accelerated gradient certain datasets. For matrix completion, enjoys smaller optimality gap, rank FW.